System and method for regulating electronic message transmissions

ABSTRACT

Systems and methods for regulating electronic messages transmissions. A message delay system is disposed between one or more first entities and a second entity within at least one network. Electronic messages are received from among at least one the first entities and the second entity at one or more message arrival times. A message delay component applies a delay to each received electronic message, based on a predefined delay time common to all of the first entities and a first entity delay offset associated with a first entity that is associated with the received message. The first entity delay offset is based on a geographical origin of the first entity relative to a geographical origin of the second entity. Each delayed message is transmitted to a designated recipient via the network, where the designated recipient is among the second entity and the first entities.

TECHNICAL FIELD

The present disclosure relates generally to improving electronic messagecommunication and, in particular, to systems and methods of controllingelectronic message transmissions routed from various geographicaloriginations with different transmission times such that the messagesarrive at a destination entity at a similar time.

BACKGROUND

Communication networks (e.g., computer networks, the internet) havebrought the world together. Through one or more communication networks,entities (via computing devices) from practically any region of theworld have the ability to exchange data with each other, regardless ofwhether the computing devices have a direct connection to each other.However, problems still exist with communication networks. Communicationnetworks may differ in network properties, for example, with respect totransmission medium for carrying signals through the network (e.g.,optical fiber, electrical cable, wireless, etc.), communicationprotocols to organize network traffic (e.g., Ethernet, wireless localarea network (WLAN), Internet Protocol Suite (i.e., transmission controlprotocol (TCP)/Internet Protocol (IP)), digital cellular standards(e.g., global system for mobile communications (GSM)), etc.), networksize, topology (e.g., bus, star, ring, mesh, fully connected, tree,etc.) and organizational intent (e.g., intranet, extranet, internet,internetwork, etc.). These differences in network properties,connections between networks, the geographic origin of the originatingcomputing device and the geographic destination of the destinationcomputing device can all impact the speed at which data is transferredbetween computing devices. Thus, two or more computing devices (forexample, at different geographical origins and/or having differentcommunication protocols) may transmit data at the same time, but thisdata (from both devices) may reach a destination computing device atdifferent times.

There is a need for improving electronic communication in communicationnetworks, to reduce the effects of differing network properties andgeographic origin on electronic message transmission, such that somecomputing devices are not placed at a disadvantage compared with othercomputing devices with respect to message transmission speed.

SUMMARY

Aspects of the present disclosure relate to systems and methods forcontrol of electronic message transmissions. The system includes one ormore first entities configured to be communicatively coupled to a secondentity via at least one network, and a message delay system disposedbetween the one or more first entities and the second entity within theat least one network. Each first entity is configured to exchangeelectronic messages with the second entity. The message delay systemincludes an input interface, a message delay component and an outputinterface. The input interface is configured to receive the electronicmessages from among at least one of the one or more first entities andthe second entity at one or more message arrival times, via the at leastone network. The message delay component is configured to apply a delayto each received electronic message. The applied delay, to each receivedelectronic message, is based on a predefined delay time common to all ofthe first entities and a first entity delay offset associated with afirst entity among the one or more first entities that is associatedwith the received electronic message. The first entity delay offset isbased on a geographical origin of the first entity relative to ageographical origin of the second entity. The output interface isconfigured to transmit each delayed message to a designated recipientvia the at least one network, where the designated recipient is amongthe second entity and the one or more first entities.

Aspects of the present disclosure also relate to systems for control ofelectronic message transmissions. The system includes a first entityconfigured to be communicatively coupled to a second entity via at leastone network, and a message delay system disposed between the firstentity and the second entity within the at least one network. The firstentity and the second entity are configured to exchange electronicmessages. The message delay system is configured to receive anelectronic message of the electronic messages from among the firstentity and the second entity, via the at least one network. Theelectronic message includes a designated recipient. The message delaysystem is also configured to control, via a message delay component,transmission of the received electronic message to the designatedrecipient via the at least one network, by delaying the receivedelectronic message according to a predetermined delay. The predetermineddelay is associated with a delay time common to entities including thefirst entity and a delay offset specifically associated with the firstentity. The predetermined delay is based on at least one of ageographical origin of the first entity, a geographical origin of thesecond entity, a message propagation time along a data path between thefirst entity and the second entity, one or more network properties ofthe first entity, one or more network properties of the second entity,one or more messaging attributes of the received electronic message, oneor more further attributes in the received electronic message and one ormore network properties of the at least one network.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a functional block diagram of an example electronic messagecommunication environment including an example message delay linesystem, according to an aspect of the present disclosure.

FIG. 2 is a functional block diagram of an example message delay linesystem according to an aspect of the present disclosure.

FIG. 3 is a flowchart diagram of an example method of assigning incomingmessages to a message queue according to delayed message departure timesassociated with the electronic message communication environment shownin FIG. 1, according to an aspect of the present disclosure.

FIG. 4 is a flowchart diagram of an example method of transmittingmessages to a destination entity via a message queue arranged accordingto a delayed message departure time associated with the electronicmessage communication environment shown in FIG. 1, according to anaspect of the present disclosure.

FIG. 5 is an example of a re-ordering of incoming messages into amessage queue such that the messages include a delayed message departuretime based on a predefined participant delay offset, according to anaspect of the present disclosure.

FIG. 6 is a flowchart diagram of an example method of transmittingmessages to a destination entity according to a delayed messagedeparture time associated with the electronic message communicationenvironment shown in FIG. 1, according to another aspect of the presentdisclosure.

FIG. 7 is a functional block diagram of an example passive clientingress point (CIP) configuration for measuring participant delay offsetassociated with the electronic message communication environment shownin FIG. 1, according to an aspect of the present disclosure.

FIG. 8 is a functional block diagram of an example active CIPconfiguration for measuring participant delay offset associated with theelectronic message communication environment shown in FIG. 1, accordingto another aspect of the present disclosure.

FIG. 9 is a functional block diagram of an example software delay lineelement (SDLE) system associated with the electronic messagecommunication environment shown in FIG. 1, according to an aspect of thepresent disclosure.

FIG. 10 is a functional block diagram of an example hardware delay lineelement (HDLE) system associated with the electronic messagecommunication environment shown in FIG. 1, according to an aspect of thepresent disclosure.

FIG. 11 is a flowchart diagram of an example method of determining andapplying a participant delay offset to messages associated with themessage delay line system shown in FIG. 2, according to an aspect of thepresent disclosure.

FIG. 12 is a flowchart diagram of an example method of determining andapplying a participant delay offset to messages associated with themessage delay line system shown in FIG. 2, according to another aspectof the present disclosure.

FIG. 13 is a functional block diagram of an example computer system,according to an aspect of the present disclosure.

DETAILED DESCRIPTION

Aspects of the present disclosure generally relate to systems andmethods of controlling electronic communication message transmissionsbetween sender and destination entities, that takes into accountgeographical distances between the sender and destination entities. Insome examples, the control mechanism includes a message delay linesystem that incorporates a sender specific delay offset, such thatmessages sent from different geographical distances at the same timearrive at the destination entity at similar times, regardless ofdifferences in message transmission times due to differences ingeographical distances, differences in transmission routes,communication mediums and other communication network properties.

In some examples, the message delay line system may include at least onesoftware-delay line element (SDLE) arrangement that may rearrangeincoming messages in a message queue according to a departure time basedon a predefined delay time common to all sender entities and a senderspecific delay offset. In some examples, the message delay line systemmay include at least one hardware delay line element (HDLE) arrangementthat may be configured to apply a hardware based delay corresponding thedeparture time to the received messages in accordance with thecorresponding sender entity. In some examples, the message delay linesystem may include one or more SDLE and HDLE arrangements. In someexamples, the sender specific delay offset may be measured based on atravel time for a data packet along a data packet path between a clientingress point (CIP) and a downstream system ingress point (DIP). In someexamples, the delay offset may be measured using a passive CIPconfiguration. In some examples, the delay offset may be measured usingan active CIP configuration including according to out-of-band and/orin-band techniques.

Turning now to FIG. 1, FIG. 1 is a functional block diagram illustratingexample electronic message communication environment 100 for controlleddelivery of electronic messages between entities, according to aspectsof the present disclosure. Environment 100 may include one or moreparticipant devices 102 (i.e., participant device 102-1, . . . ,participant device 102-N, where N is an integer greater than or equal to1), message delay line system 104 and at least one downstream system106. Each of participant devices 102, message delay line system 104 anddownstream system 106 may comprise one or more computing devices,including a non-transitory memory storing computer-readable instructionsexecutable by a processing device to perform the functions describedherein. Although the description herein describes environment 100 havingthree or more participant devices 102, in some examples, environment 100may include one participant device 102 (i.e., where N is equal to 1).

Participant devices 102, message delay line system 104 and downstreamsystem(s) 106 may be communicatively coupled via one or more networks(not shown). The at least one network may include, for example, aprivate network (e.g., a local area network (LAN), a wide area network(WAN), intranet, etc.) and/or a public network (e.g., the internet).Although FIG. 1 illustrates one downstream system 106, in some examples,environment 100 may include more than one downstream system 106, eachelectronically coupled to message delay line system 104 via at least onenetwork.

Participant devices 102 may be configured to transmit electroniccommunication messages (also referred to herein as “electronic messages”or “messages”) directed to downstream system 106. For example,participant device 102-1 (associated with Participant 1 (P1)) maytransmit one or more messages__(P1), participant device 102-2(associated with Participant 2 (P2)) may transmit one or moremessages__(P2) and participant device 102-N (associated with ParticipantN (PN)) may transmit one or more messages__(PN). Although not shown,downstream system 106 may similarly transmit electronic messagesdirected to one or more of participant devices 102 (and/or to anotherdownstream system (not shown)).

In some examples, message delay line system 104 may be configured toreceive messages that are explicitly directed to message delay linesystem 104 (with an indication in the message of a final destination),and message delay line system 104 may transmit the messages to theirfinal destination (e.g., downstream system 106) according to acorresponding departure time delay. In some examples, message delay linesystem 104 may intercept messages that are explicitly directed to adestination other than message delay line system 104 (e.g., downstreamsystem 106), and message delay line system 104 may transmit theintercepted messages to their destination(s) according to a departuretime delay.

Participant devices 102 may comprise a desktop computer, a laptop, asmartphone, tablet, or any other user device known in the art. Aparticipant may interact with participant device 102, for example, via agraphical user interface (not shown) displayed on any type of displaydevice including a computer monitor, a smart-phone screen, tablet, alaptop screen or any other device providing information to aparticipant. Participant devices 102 may include any suitable userinterface, user input component(s), output component(s), andcommunication component(s) for creation and transmission and receipt ofelectronic communication messages. The electronic communication messagesmay include, without being limited to, instant messages, personalmessages, text messages and email.

Each participant device 102 may be located within a geographical origin108. For example, participant device 102-1 may be located withingeographical origin 108-1, participant device 102-2 may be locatedwithin geographical origin 108-2 and participant device 102-N may belocated within geographical origin 108-N. FIG. 1 illustrates one examplewhere each participant device 102 is within a different geographicalorigin 108 (e.g., 108-1, 108-2, 108-N). In some examples, two or moreparticipant devices 102 may be located within a same geographical origin108. In some examples, geographic origin 108 may represent an actuallocation of a particular participant device 102. In some examples,geographic origin 108 may represent an area in which the particulardevice 102 is located (e.g., a 10 km radius). In some examples, one ormore of geographical origins 108 may be co-located with message delayline system 104 and/or downstream system 106.

Each geographical origin 108 may be associated with a predeterminedtravel time associated with routing an electronic message fromgeographical origin 108 to message delay line system 104 (and,ultimately to downstream system(s) 106). The predetermined travel timefor an electronic message from a particular geographic origin (e.g.,108-1) may be based on, for example, a geographical distance betweenparticipant device 102 and message delay system 104, a communicationmedium(s) through which the electronic message travels, any networkpropagation delay(s), other network parameters (e.g., communicationprotocol, network size, topology, organizational intent), etc. In someexamples, a geographical origin 108 may be co-located with downstreamsystem(s) 106 and may have minimal travel time. In some examples, ageographical origin 108 may be differently located and may have a longertravel time. Thus, without message delay line system 104, two messages(e.g., message__(P1) and message__(P2) transmitted at the same time fromdifferent geographical origins (e.g., origin 108-1, for example, inFrankfurt, Germany and origin 108-2, for example, in Chicago, Ill., USA)may reach downstream system 106 (for example, located in Mahwah, N.J.,USA) at different times.

Downstream system 106 may be configured to receive electronic messagesfrom among participant device(s) 102, as well as from one or more otherdownstream system (not shown). In some examples, downstream system 106may comprise an electronic exchange system. In such examples, electronicmessages from participant device(s) 102 may include, for example, orderdata (e.g., bid and/or offer data) for one or more assets. Electronicmessages from downstream system 106 may include, for example, marketdata information, transaction information, etc. In one particularnon-limiting implementation, an electronic exchange system may refer toan electronic asset exchange system or device such as a commoditiesexchange, a futures execution facility, an options exchange, a cashequities exchange, a swap execution facility, an electronic transactionexecution venue or any other type of an exchange venue known in the art.In some examples, an electronic exchange system may refer to a simpledata transfer/exchange system. In some examples, downstream system 106may include any system or device configured to process information inmessages, where the information in the electronic messages may be timesensitive. For example, downstream system 106 may include an onlinetravel reservation venue, an online retail shopping site, an electronicauction, etc.

Downstream system 106 may comprise one or more processors configured toexecute instructions stored in a non-transitory memory (such as shown inFIG. 13). Downstream system 106 may be embodied on a single computingdevice, while in other embodiments, downstream system 106 may refer to aplurality of computing devices housed in one or more facilities that areconfigured to jointly provide local or remote computing services to oneor more participants or participant devices 102. In general, downstreamsystem 106 may send and receive data from participant devices 102, dataservers, or any other type of computing devices or entities over theinternet, over a Wi-Fi connection, over a cellular network or via anyother wired or wireless connection or network known in the art.

In general, the type of data included in the electronic communicationmessages between participant devices 102 and downstream system 106 maydepend on the particular implementation. In general, downstream system106 may include any system (e.g., one or more computing devices) forperforming one or more processes based on data in the received messages.Message delay line system 104 may be implemented, for example, with anydownstream system where it may be beneficial to delay messages fromparticipants having different travel times, such that the electronicmessages arrive at the downstream system 106 at a similar or same time(or such that electronic message(s) from downstream system 106 to one ormore participant device(s) 102 may be delayed).

Message delay line system 104 may be configured to receive incomingmessages from among participant devices 102 and/or another downstreamsystem, determine a delayed departure time for each message, and controldelivery of each message to a destination (e.g., downstream system 106or participant device(s) 102) based on the delayed departure time. Inthe examples below, message delay line system 104 is described asreceiving incoming messages from participant devices 102 andtransmitting these messages to downstream system 106 after re-orderingthe messages by respective delayed departure times. It is understoodthat message delay line system 104 may be configured to implement asimilar delayed departure time transmission procedure for outgoingmessages from downstream system 106 to one or more participant devices102.

Each incoming electronic message to message delay line system 104 mayindicate a message sender (e.g., a participant device 102 such asparticipant device 102-1 or downstream system 106), the geographicorigin 108 of the sender (e.g., geographic origin 108-1) and adestination (e.g., message recipient). In some examples, the electronicmessages may be encrypted. The sender and the geographical origin may beused by message delay line system 104 to determine the message departuretime delay.

The (delayed) departure time (T_(departure)) may be based on a commondelay time (T_(delay)) and a (negative) participant delay offset(T_(participant)), such that:T _(departure) =T _(arrival) +T _(delay) −T _(paracipant)  (1)The common delay time (T_(delay)) is applied equally to each participantregardless of geographic origin 108. The participant delay offset isbased on the particular geographic origin 108 of a particularparticipant device 102 (e.g., geographic origin 108-1 of participantdevice 102-1).

Each delay (e.g., T_(delay) and T_(participant)) may be determined basedon a geographical distance between participant device 102 and downstreamsystem 106, a communication medium(s) through which the electronicmessage travels, as well as any network propagation delay(s). Forexample, a 10 μs delay in time may be equivalent to 1 km of opticalfiber length or 3 km of light in vacuum propagation. In the examples, itis assumed that message delay line system 104 is co-located withdownstream system 106, such that the distance to downstream system 106is the same as the distance to message delay line system 104. Inexamples where message delay line system 104 and downstream system 106are not co-located, it is understood that message delay line system 104may factor any additional delay to downstream system 106 in itsdetermination of the departure time.

The common message delay (T_(delay)) may represent a fair access areafor participant devices located within a particular area centered on alocation of downstream system 106. For example, a common T_(delay) of500 μs corresponds to a distance (e.g., radius) of 150 km that it takeslight travels in a vacuum. Thus, adding T_(delay) to an incoming messagecorresponds to moving all message senders away from downstream system106 by the same distance (e.g. 150 km).

The participant delay offset (T_(participant)) corresponds to ageographical delay credit (GDC) that may be assigned to participantdevices, to compensate for submitting the electronic message for ageographic origin 108 that is not co-located with downstream system 106(e.g., participant device 102-1 may be farther away from downstreamsystem 106 compared to participant device 102-2 and its electronicmessage would be subject to a longer travel time). The participant delayoffset may range from 0 μs (e.g., when participant device 102 isco-located with downstream system 106) to a maximum delay offset of(T_(delay)). In some examples, the GDC may be based on a networkpropagation delay—(the amount of time light takes to travel through thecommunication medium for the geographical distance between geographicalorigin 108 and downstream system 106). In some examples, the GDC mayalso be adjusted for some message senders (e.g., another downstreamsystem such as another electronic exchange), so that data in thesemessages can be processed faster. Measurement and application of theparticipant delay offset for various system configurations is describedfurther below with respect to FIGS. 7-12.

Accordingly, the departure time for a participant device 102 co-locatedat downstream system 106 may be based on T_(delay) only (becauseT_(participant) is set to zero). A participant device 102 locatedfarther away from downstream system 106 may include a participant delayoffset (T_(participant)) that reduces the common delay time applied tothe incoming message arrival time. When the distance from downstreamsystem is great enough, the participant delay offset equals the commondelay time (i.e., T_(participant)=T_(delay)) and no delay may be appliedto the message departure time. As another example, when T_(delay) is thesame for all arriving messages, and T_(participant) is zero for allincoming messages, all messages leave system 104 in the order ofarrival. When, however, T_(delay) stays the same but T_(participant)varies from message to message (per participant device 102), system 104may potentially re-order the incoming messages. For example, earlierarrivals by participant device 102-1 may be transmitted from system 104after a later arrived message from participant device 102-2).

Thus, message delay line system 104 may control the departure time ofmessages based on the geographical origin(s) 108 of participant devices102, with less delay applied to farther geographical origins 108 andmore delay applied to closer geographical origins 108 relative todownstream system 106. In this manner, each electronic message (that istransmitted at the same time) may be received by downstream system 106at a same or similar time, regardless of the geographic origin 108 (androuting path). Accordingly, the departure time (T_(departure))incorporates a smart delay that may eliminate any proximity advantage byparticipant devices 102. Thus, message delay line system 104 may beconfigured to mitigate any co-location advantages by some participantdevices 102 compared to other participant devices at differentgeographic origins 108.

Geographic origins 108, message senders (e.g., participant devices 102,downstream system 106, other downstream system(s)), communicationmediums and network propagation delays may be predetermined by messagedelay line system 104 and used to set a common (predefined) delay timeT(_(delay)) for all senders, and to set a participant delay offset(T_(participant)) associated with each message sender (e.g., eachparticipant device 102, downstream system 106, another downstreamsystem). The participant delay offsets for each sender and geographicorigin 108 may be stored, for example, in a look-up table stored in adatabase (such as database 216 shown in FIG. 2).

In operation, message delay line system 104 may time-stamp each incomingmessage, identify a sender of the time-stamped message (e.g.,participant device 102-1) and identify a geographic origin of themessage (e.g., geographic origin 108-1). Message delay line system 104may then determine the participant delay offset (T_(participant)) basedon the identified information (e.g., sender and geographic origin) andthe common delay time (T_(delay)) such as via a look-up table. System104 may then determine the departure time for the timestamped messagebased on equation (1).

Message delay line system 104 may assign the timestamped message to aposition in a message queue (e.g., message queue 206 shown in FIG. 2)according to the departure time. As discussed further below with respectto FIG. 5, by arranging the messages in the message queue according todeparture time, messages may be ordered differently in the message queuecompared with message arrival time (e.g., re-ordered), and may leave themessage queue in a different order compared to the order in which themessages are received by message delay line system 104. System 104 maytransmit a message in the message queue to the destination (e.g.,downstream system 106) based on the departure time (T_(departure)).

It is understood that system 104 may include two or more message queues.For example, one message queue may be assigned to incoming messages fromparticipant devices 102 (and in some examples, other downstreamsystem(s)) for transmission to downstream system 106. A separate messagequeue may be assigned to outgoing messages from downstream system 106for transmission to participant devices 102 (as well as any to any otherdownstream systems).

In some examples, message delay line system 104 and downstream system106 may be embodied on a single computing device. In other examples,message delay line system 104 and downstream system 106 may refer to twoor more computing devices distributed over several physical locations,connected by one or more wired and/or wireless links. It should beunderstood that message delay line system 104 refers to a computingsystem having sufficient processing and memory capabilities to performthe following specialized functions. Message delay line system 104 isdescribed further below, according to FIG. 2.

FIG. 2 is a functional block diagram of example message delay linesystem 104, according to an aspect of the present disclosure. Messagedelay line system 104 may include input interface 202, wall clock 204,at least one message queue 206, output interface 208, timestamp module210, message departure time module 212, message queue controller 214 andstorage 216, which may communicate with each other via data and controlbus 218. Message queue controller 214 may include, for example, aprocessor, a microcontroller, a circuit, software and/or other hardwarecomponent(s) specially configured to control operation of inputinterface 202, wall clock 204, message queue(s) 206, output interface208, timestamp module 210, message departure time module 212 and storage216.

Input interface 202 may represent any electronic device or applicationon an electronic device configured to receive incoming messages fromvarious sender entities (e.g., participant devices 102, downstreamsystem 106, other downstream system(s)) via at least one network. Insome examples, input interface 202 may be configured to securelycommunicate with one or more of the sender entities. In some examples,input interface 202 may be configured to communicate with various senderentities via a wired or wireless connection.

Timestamp module 210 may be configured to apply a timestamp to anincoming message received at input interface 202, via wall clock 204.Wall clock 204 may include any suitable local clock circuit foridentifying a time of receipt of the incoming message by system 104 viainput interface 202.

Message departure time module 212 may be configured to determine adeparture time for the timestamped message, based on equation (1).Message departure time module 212 may identify a participant 102 and ageographical origin 108 from the timestamped message, and may determinethe common time delay and the participant delay offset associated withthe identified information (participant and geographical origin) frompredefined information stored in storage 216. The retrieved informationfrom storage 216 may be used to determine the departure time for thetimestamped message.

Message queue controller 214 may be configured to assign each incoming(timestamped) message to message queue(s) 206 based on the departuretime determined by module 212. Message queue controller 214 may orderincoming messages, for example, in an order such that a message with ashortest departure time is positioned at the head of message queue 206and a message with a longest departure time is positioned at the bottomof message queue 206. In some examples, message queue controller 214 mayinclude more than one message at a same position of message queue(s)206, for example, when two or more messages have a same departure time.Message queue 206 may be configured with any suitable architecture forstoring electronic messages and providing an asynchronous communicationprotocol.

Message queue controller 214 may also be configured to remove one ormore messages in message queue(s) 206, transmit the removed message(s)to a destination entity (e.g., downstream system 106, participantdevice(s) 102, another downstream system) via output interface 208, andrearrange (e.g., update) the entries in message queue(s) 206. Forexample, message queue controller 214 may compare the current wall clocktime from wall clock 204 with the departure time for the message(s) atthe head of message queue 206. When the current wall clock time(T_(current)) is greater than the departure time (T_(departure)) for thehead message(s), the head message(s) may be removed from message queue206.

Output interface 208 may represent any electronic device or applicationon an electronic device configured to output the transmitted message(s)(at the departure time) to destination entities (e.g., downstream system106, participant device(s) 102, another downstream system) via at leastone network. In some examples, output interface 208 may be configured tocommunicate with destination entities via a wired or wirelessconnection. In some examples, output interface 208 may be configured tosecurely communicate with one or more of the destination entities. Insome examples, input interface 202 and output interface 208 mayrepresent separate interfaces. In some examples, input interface 202 andoutput interface 208 may represent a single input/output (I/O)interface.

Storage 216 may include any suitable non-transitory storage medium andmay be configured to store predefined participant delay offsetsT_(participant) (e.g., GDCs) associated with each participant device102, as well as any other downstream systems. Storage 216 may also storea predetermined common delay time (T_(delay)) that may be equallyapplied to all participant devices 102 (and other downstream systems).

In some examples, message delay line system 104 may include one or morehardware delay line elements (HDLEs) 220. HDLE 220 may represent ahardware-based device having one or more hardware delay components (suchas hardware delay component 1006 of HDLE 1004 in FIG. 10) configured toapply a predefined delay to an electronic message. Examples of suitablehardware delay components may include, without being limited to, a spoolof electrical wire, a spool of optical fiber and an electronic devicecomprising a non-transitory memory. The predefined delay of the hardwarecomponent may correspond to a participant delay offset. HDLE 220 may beconfigured with a plurality of predefined delays associated withdifferent participant delay offsets. In operation, participant devices102 may be assigned to different hardware components. For example,participant device 102-1 may be assigned to hardware component 1006-1having participant 1 delay offset and participant device 102-2 may beassigned to hardware component 1006-2 having a participant 2 delay(e.g., a different predefined delay). In some examples, the predefineddelay may correspond to the departure time (T_(departure)) that is thecombination of the common delay time (T_(delay)) and the participantdelay offset (T_(participant))

In some examples, HDLE 220 may be provided prior to input interface 202message delay line system, such that incoming messages are subjected toa hardware-based delay prior to the application of any software-baseddelay by message departure time module 212 and message queue controller214. In some examples, HDLE element 220 may be placed in system 104after output interface 208, such that outgoing messages are subjected toa hardware-based delay after the application of any software-based delayby message departure time module 212 and message queue controller 214.In some examples, the hardware-based delay by HDLE 220 may replace atleast some of the functions of message departure time module 212, andmessage queue controller 214 may position messages in message queue(s)206 in the order in which the messages are received (based on the outputfrom timestamp module 210). Thus, in some examples, message delay linesystem 104 may only apply a software-based delay (i.e., system 104 maynot include HDLE element 220), may apply a combination of software andhardware-based delay (i.e., system 104 may include element 220 at inputinterface 202 and/or output interface 208) or may apply only a hardwarebased delay (i.e., HDLE 220 may replace at least some of the functionsof message departure time module 212 in system 104).

Some portions of above description describe the embodiments in terms ofalgorithms and symbolic representations of operations on information.These algorithmic descriptions and representations are commonly used bythose skilled in the data processing arts to convey the substance oftheir work effectively to others skilled in the art. These operations,while described functionally, computationally, or logically, areunderstood to be implemented by computer programs or equivalentelectrical circuits, microcode, or the like. Furthermore, it has alsoproven convenient at times, to refer to these arrangements of operationsas modules, without loss of generality. The described operations andtheir associated modules may be embodied in specialized software,firmware, specially-configured hardware or any combinations thereof.

Those skilled in the art will appreciate that message delay line system104 may be configured with more or less modules to conduct the methodsdescribed herein with reference to FIGS. 3, 4, 6, 11 and 12. Asillustrated in FIGS. 3, 4, 6, 11 and 12, the methods shown may beperformed by processing logic that may comprise hardware (e.g.,circuitry, dedicated logic, programmable logic, microcode, etc.),software (such as instructions run on a processing device), or acombination thereof. In one embodiment, the methods shown in FIGS. 3, 4,6, 11 and 12 may be performed by one or more specialized processingcomponents associated with components 202-218 of message delay linesystem 104 of FIG. 2.

FIG. 3 illustrates a flowchart diagram of an example method of assigningincoming messages to message queue 206 according to delayed messagedeparture times associated with electronic message communicationenvironment 100 shown in FIG. 1, according to an aspect of the presentdisclosure. FIG. 4 is a flowchart diagram of an example method oftransmitting messages to a destination entity via message queue 206arranged according to a delayed message departure time associated withelectronic message communication environment 100 shown in FIG. 1,according to an aspect of the present disclosure. FIGS. 3 and 4 aredescribed with reference to FIGS. 1 and 2. FIGS. 3 and 4, collectively,illustrate a method for controlling electronic messages transmissionsrouted from various geographical origins such that the messages arriveat a destination entity at similar times. FIG. 3 illustrates an exampleof message delay system 104 without HDLE 220. An example of messagedelay system 104 with HDLE 220 is described further below in FIG. 6.

At step 300, message delay line system 104 may receive an electronicmessage from among one or more sender entities (for example, from amongparticipant(s) device 102, downstream system 106 or another downstreamsystem) via input interface 202. At step 302, timestamp module 210 mayadd a timestamp to the received electronic message based on a currentwall clock time, for example, via wall clock 204.

At step 304, message departure time module 212 may identify aparticipant and a message origin included in the electronic message(e.g., from information in the message header). In some examples, thesender entity of the message may correspond to the participant (forexample, for electronic messages sent from participant devices 102 todownstream system 106). In some examples, the destination entity of themessage may correspond to the participant (for example, for electronicmessages sent from downstream system 106 to participant devices 102, fortransmission at a departure time associated with geographical origin 108of the participant device 102).

At step 306, message departure time module 212 may determine aparticipant delay offset (T_(participant)) based on the identifiedinformation, for example, by querying storage 216. At step 308, messagedeparture time module 212 may determine the departure time for themessage, based on the timestamped arrival time (step 302), the common(predefined) delay time (T_(delay)) and the participant delay offset,according to equation 1.

At step 310, message queue controller 214 may assign the message to aparticular position in message queue(s) 206 according to the messagedeparture time (step 308).

At step 400, message queue controller 214 may receive the current wallclock time (T_(current)), for example, via wall clock 204. At step 402,message queue controller 214 may compare a departure time(T_(departure)) for a message positioned at the head of message queue206 to the current wall clock time.

At step 404, message queue controller 214 may determine whether thecurrent wall clock time (T_(current)), is greater than the messagedeparture time (T_(departure)). When T_(current) is not greater than(i.e., less than or equal to) T_(departure), step 404 proceeds to step400.

When T_(current) is greater than T_(departure), step 404 proceeds tostep 406. At step 406, message queue controller 214 removes the messageassociated with the departure time from a head position of message queue206. At step 408, message queue controller 214 may transmit the(removed) message to a destination entity, for example, via outputinterface 208. At step 410, message queue controller 214 may update thearrangement of messages remaining in message queue(s) 206.

Referring next to FIG. 5, an example is shown of a re-ordering ofincoming messages into message queue 206 according to a delayed messagedeparture time that is based on a participant delay offset, according toan aspect of the present disclosure. In the example, participant device102-1 (P1) is co-located at downstream system 106. Thus, P1'sparticipant delay offset (T_(participant)(P1)) may be set to 0 μs.Participant device 102-2 (P2) may have a geographical origin 108-2 thatis 1 km away from downstream system 106. P2's participant delay offsetT_(participant)(P2) may be set to 10 μs. Furthermore, in this example,the common delay time (T_(delay)) may be set to 100 μs.

At Wall Clock 0, system 104 may receive an electronic message from P1,and may determine P1's message departure time (T_(departure)) fromequation (1) as 100 μs (i.e.,: 0+100−0=100 μs). If, at Wall Clock 5,system 104 receives a message from P2, system 104 may determine P2'smessage departure time (T_(departure)) as 95 μs (i.e.,: 5+100−10=95 μs).As shown in FIG. 5, the messages for P1 and P2 are placed in messagequeue 206 according to their respective departure times. Assuming thatno other messages are received, then, at Wall Clock 95 μs, P2's messageis the first message to be transmitted by system 104. At Wall Clock 100μs, system 104 then transmits P1's message.

Assuming that system 104 subsequently receives a message from P2 at WallClock 11, system 104 may determine T_(departure) for P2's second messageas 101 μs (i.e., 11+100−10=101 μs). If no other messages are received,system 104 transmits P2's second message at wall clock 101 (after P1'smessage that is transmitted at Wall Clock 100).

In contrast, without the participant-specific delay offset, the order ofthe messages that would be received by downstream system 106 would bethe order in which the messages are received (e.g., the left table). Forexample, P1 at Wall Clock 100 (T_(arrival)+T_(delay)), P2 at Wall Clock105 (corresponding to arrival time 5 μs) and P2 at 111 (corresponding toarrival time 11 μs).

FIG. 6 illustrates a flowchart diagram of an example method of assigningincoming messages to message queue 206 such that the messages include adelayed message departure times associated with electronic messagecommunication environment 100 shown in FIG. 1, according to an aspect ofthe present disclosure. In particular, FIG. 6 illustrates one or moreexamples of message delay system 104 including HDLE 220. FIG. 6 isdescribed with reference to FIGS. 1 and 2.

At step 600, HDLE 220 of message delay line system 104 may receive anelectronic message from among one or more sender entities (for example,from among participant(s) device 102, downstream system 106 or anotherdownstream system). At step 602, HDLE 220 may apply a predefined delayto the received electronic message. The predefined delay may include theparticipant delay offset or the departure time. At step 604, timestampmodule 210 may receive the delayed message (from HDLE 220 via inputinterface 202) and add a timestamp to the received electronic messagebased on a current wall clock time, for example, via wall clock 204.

At step 606, message departure time module 212 may identify aparticipant and a message origin included in the electronic message(e.g., from information in the message header). In some examples, thesender entity of the message may correspond to the participant (forexample, for electronic messages sent from participant devices 102 todownstream system 106). In some examples, the destination entity of themessage may correspond to the participant (for example, for electronicmessages sent from downstream system 106 to participant devices 102, fortransmission at a departure time associated with geographical origin 108of the participant device 102).

At optional step 608, message departure time module 212 may determine anadditional participant delay offset based on information in the receivedmessage. At optional step 610, message departure time module 212 maydetermine the departure time for the message. For example, the departuretime may be determined based on the timestamped arrival time and thecommon (predefined) delay time (T_(delay)) (because the participantdelay is applied by HDLE 220). In another example, the departure timemay further include the optional additional participant delay offset(step 608). In some examples, step 610 may not be performed, forexample, when HDLE 220 applies the departure time delay to the incomingmessage (at step 602).

At step 612, message queue controller 214 may assign the message to aparticular position in message queue(s) 206. In some examples, themessage may be assigned to the message queue(s) 206 in the order it isreceived (when HDLE 220 applies a predefined delay corresponding to thedeparture time. In some examples, the message may be assigned to themessage queue(s) 206 according to the message departure time (determinedat optional step 610). Step 612 may proceed to step 400 in FIG. 4.

In some examples (not shown in FIG. 6), steps 604-612 may be performedfirst (without step 600) and HDLE 220 may apply a hardware delay to theoutgoing message (e.g., after step 408 in FIG. 4). In some examples,steps 600-612 may be performed, and an additional hardware delay may beapplied by HDLE 220 to the outgoing message (e.g., after step 408 inFIG. 4).

Referring next to FIGS. 6-12, examples of system configurations andmethods for determining and applying the participant delay offset inenvironment 100 are described. In general, to determine the participantdelay offset, a packet data path between participant 102 and downstreamsystem 106 may be considered. In particular, the travel time for apacket along a packet data path between a client ingress point (CIP) anda downstream system ingress point (DIP) may be determined. The CIP maybe defined as any point beyond which a participant has no control of thepacket data path. The DIP may be defined as the point closest todownstream system 106, past which all participant devices 102 have equalaccess time to message delay system 104.

FIGS. 7 and 8 illustrate functional block diagrams of example passiveand active CIP configurations for measuring the participant delayoffset.

Referring to FIG. 7, an example passive CIP configuration 700 formeasuring participant delay offset is shown. Passive CIP configuration700 may include patch panel 708 having one or more passive CIPs 710coupled to participant space 702 and patch panel 712 having one or moreDIPs 714 coupled to downstream system space 706. CIP 710 (via patchpanel 708) and DIP 714 (via patch panel 712) may be coupled to carrierspace 704, such that client space 702 and downstream system space 706may be configured to exchange data packets (e.g., via electronicmessages) via carrier space 704. In general, patch panels 708, 712 eachrepresent a mounted hardware assembly comprising one or more ports in alocal area network (LAN) used to connect and manage incoming andoutgoing LAN cables.

The data path between CIP 710 and DIP 714 in carrier space 704 isdefined herein as a total cross-connect (TCC). The time to send a packetalong the TCC in carrier space 704 may be represented as a TCCpropagation delay. The participant delay offset may be determined bymeasuring the TCC propagation delay.

In general, a passive client does not have any active CIP component. Asa result, measurement of the TCC propagation delay may be performedunder the control of downstream system space 706 and/or carrier space704. In operation, the TCC propagation delay may be measured, forexample, using an optical delay measurement device or network testersuch as, without being limited to a T-1 bit error rate detector(T-BERD), at the time of provisioning the participant, and may assumedto be constant. As another example, the TCC propagation delay may bemeasured based on packet decoding techniques, such as using a precisiontime protocol (PTP), for example as defined in the IEEE 1588 standard.The measured TCC propagation delay may be associated with theparticipant, and stored as the participant delay offset, for example, instorage 216 (FIG. 2).

Non-limiting examples of passive CIP 710 may include a wiring closest onthe client premises or a patch panel port in a client (participant)cabinet in a data center. The port may represent an official demarcationline in a data center, and may require a Letter of Authorization (LOA),signed by the client (participant) and the owner of a patch panel porton the downstream space 706 side or carrier space 704 side for across-connect line to a next patch panel. In another example, passiveCIP 710 may include a patch panel in a “meet-me-room” on the clientpremises, with a similar LOA arrangement. In general, a patch panel mayrepresent colocation of at least a portion of the participant'sequipment with downstream system 106, whereas a “meet-me-room” mayrepresent participant equipment located on some other premises thandownstream system 106, such as client data center, with a hand-off to aclient-independent entity in the “meet-me-room.” In general passive CIP710 may represent any point from which the TCC propagation delay may befixed and not under client control.

Referring to FIG. 8, an example active CIP configuration 800 formeasuring participant delay offset is shown. Active CIP configuration800 may include active CIP component 808 coupled to participant space802 and patch panel 812 having one or more DIPs 814 coupled todownstream system space 806. Active CIP component 808 and DIP 814 (viapatch panel 812) may be coupled to carrier space 804, such that clientspace 802 and downstream system space 806 may be configured to exchangedata packets (e.g., via electronic messages) via carrier space 804.Patch panel 812 is similar to patch panel 712 (described above).

Active CIP component 808 may include any device, card and/or circuitboard through which data packets from participant space 802 pass throughand which may be configured to communicate with downstream system space806 (and which may be controlled by downstream system 106). In general,active CIP component 808 may include any device acting like a“bump-in-the-wire,” through which participant data packets may passthrough, and after which the participant has no control of the datapath. In some examples, active CIP component 808 may be configured toaffix time stamps to the data packets.

Non-limiting examples of active CIP component 808 may include a networkinterface card (NIC) (controlled by downstream system 106) on aparticipant-side server and a participant-side server (controlled bydownstream system 106). In one non-limiting example, downstream system106 may include an exchange, and active CIP component 808 may include anexchange-controlled NIC on the participant-side server and/or anexchange controlled client-side server such as a trading gateway.

For active CIP configuration 800, the TCC propagation delay may bemeasured by out-of-band techniques and/or in-band techniques. Forexample, with out-of-band techniques, message delay line system 104 mayperiodically ping active CIP component 808, in order to measure the TCCpropagation delay, associate the measured TCC propagation delay with theparticipant and store the measured TCC propagation delay as theparticipant delay offset in storage, such as storage 216 (FIG. 2). Within-band techniques, active CIP component 808 may time-stamp each packettransmitted to message delay line system 104, and message delay linesystem 104 may compare the difference in time-stamps (between thetime-stamp of active CIP component 808 and the time-stamp applied bytime-stamp module 210 (FIG. 2) of message delay line system 104.

In general, passive CIP configuration 700 may be more efficient thanactive CIP configuration 800, because no client-side components arerequired for installation, and only one initial TCC propagation delaymeasurement may be obtained and used during message delay application.Active CIP configuration 800 represents a more complex configuration,including installation of an active CIP component 808 on the participantpremises. Active CIP configuration 800, however, may provide moreflexibility for environments where it is difficult to control the datapacket path between the CIP and the DIP.

Referring next to FIG. 9, in some examples, message delay line system104 may represent a software delay element (SDLE) configured to apply asoftware-based delay to incoming messages on a participant-specificbasis, according to the common time delay and the correspondingparticipant delay offset (eq. 1). In FIG. 9, a functional block diagramof an example SDLE system 900 comprising message delay line system 104is shown, according to an embodiment of the present disclosure. System900 may include patch panel 902, switch 904 and message delay linesystem 104. Patch panel 902 may include one or more DIPs (e.g., DIP1,DIP2 and DIP3) associated with downstream system 106. Patch panel 902 issimilar to patch panels 712, 712 of downstream system spaces 706, 806.

SDLE system 900 may be configured to operate with passive CIPconfiguration 700 (FIG. 7) and/or active CIP configuration 800 (FIG. 8).In operation, messages from participant space 702, 802 that aretransmitted over carrier space 704, 804, are received by patch panel 902via one or more DIPs. The received messages may be routed to networkswitch 904 and then routed to message delay line system 104. Messagedelay line system 104 may apply a software-based delay and then transmitthe outgoing delayed messages to downstream system 106 (as discussedabove with respect to FIGS. 3 and 4).

For passive CIP configuration 700, software delay line system 106 mayretrieve the participant delay offset from storage 216 that is measuredat the time of participant provisioning, to determine the messagedeparture time. For active CIP configuration 800, software delay linesystem 106 may use out-of-band and/or in-band techniques to periodicallyand/or continually measure the participant delay offset, to determinethe message departure time.

Referring next to FIG. 10, in some examples, message delay line system104 may include an HDLE configured to apply a hardware-based delay toincoming messages on a participant-specific basis, according to thecommon time delay and the corresponding participant delay offset. FIG.10 illustrates a functional block diagram of an example HDLE system1000, according to another embodiment of the present disclosure. System1000 may include patch panel 102, HDLE 1004 and network switch 904. Insome examples, system 1000 may include additional components of messagedelay line system 104 (such as message queue(s) 206). Patch panel 1002may include one or more DIPs (e.g., DIP1, DIP2 and DIP3) associated withdownstream system 106. Patch panel 1002 is similar to patch panel 712 ofdownstream system space 706.

In some examples, HDLE system 1000 may be configured to operate withpassive CIP configuration 700 (FIG. 7). In operation, messages fromparticipant space 702 transmitted over carrier space 704 is received bypatch panel 1002 via one or more DIPs. The received messages may berouted to hardware components 1006, where each hardware component (e.g.,components 1006-1, 1006-2, 1006-3) may be configured with differentpredefined delay times associated with different participants. In FIG.10, hardware components 1006 are represented as spools of wire. It isunderstood that this represents a non-limiting example, and that otherconfigurations of HDLE components 1006 may be used (as discussed above).In some examples, the predefined delay times of elements 1006 may beconfigured/selected to correspond to a participant delay offset measuredduring provisioning of an associated participant. In some examples, thepredefined delay may be based on the common delay time and theparticipant delay offset (eq. 1). HDLE 1004 and patch panel 1002 may bearranged in a fixed configuration, for example, arranged at the time ofparticipant provisioning. The delayed messages may be routed from HDLE1004 to network switch 1008, and network switch 1008 may then controlrouting of the outgoing delayed messages to downstream system 106.

Referring next to FIG. 11, a flowchart diagram is shown of an examplemethod of determining and applying a participant delay offset tomessages associated with message delay line system 104, according to anembodiment of the present disclosure. FIG. 11 is described withreference to FIGS. 1, 2, 7 and 8. FIG. 11 illustrates determination ofthe participant delay offset for both an example passive CIPconfiguration 700 and an active CIP configuration 800 using out-of-bandTCC delay measurement techniques.

At step 1100, a participant record for a particular participant device102 (e.g., participant device 102-1) may be created, for example, at atime of provisioning the particular participant device 102. For example,message delay line system 104 (e.g., controller 214) may create aparticipant record for a particular participant device 102 (e.g.,participant device 102-1), and store the participant record in storage216. As another example, downstream system 106 may create theparticipant record, and may provide the participant record to storage216 of message delay line system 104.

At step 1102, internet protocol (IP) attributes associated with theparticipant device 102 (e.g., participant device 102-1) may bedetermined, at the provisioning time, and stored in the participantrecord (created at step 1100). For example, IP version 4 (v4) attributessuch as source IP address (src), subnet mask (subnet), destination IPaddress (dest IP) and destination port (dest port) may be determined andstored in the participant record. For example, message delay line system(e.g., controller 214) may determine the IP attributes for theparticipant device 102. As another example, downstream system 106 maydetermine the IP attributes and store the attributes in the participantrecord (in storage 216). In one example, the participant record maystore the IP attributes in a format such as: src subnet/dest IP/destport. At step 1102, an application platform (e.g., at downstream system106 or message delay line system 104) may also allocate a unique port(e.g., IP:PORT) per participant associated with the applicationplatform.

At step 1104, the TCC propagation delay for the participant device 102may be initially measured at the provisioning time (e.g., prior tomessage transmissions). For example, in passive CIP configuration 700,the TCC propagation delay may be measured, e.g., using an optical delaymeasurement technique and/or a delay measurement via packet decoding. Asanother example, in active CIP configuration 800 with an out-of-bandtechnique, downstream system 106 or message delay line system 104 (e.g.,message departure time module) may ping active CIP component 808 todetermine the TCC propagation delay.

At step 1106, the measured TCC propagation delay (step 1104) may bestored in the participant record (in storage 216) as the participantdelay offset, for association with the particular participant. Inpassive CIP configuration 700, the initially stored participant delayoffset may be considered a predetermined participant delay offset, andthis predetermined (fixed) value may be used during the message delayprocess by message delay line system 104 (described above in FIGS. 3 and4). In some examples, steps 1100-1106 may represent provisioning stepsfor each participant device 102, prior to participation in messageexchange in environment 100.

At step 1108, message delay line system 104 (e.g., via controller 214)may create a filter to match IP attributes to a particular participantand participant record. At step 1110, message delay line system 104, viainput interface 202, may receive an electronic message from amongparticipant devices 102. At step 1112, message delay line system 104 mayapply the created filter (step 1108) to the received message, to inspectthe IP attributes, identify the associated participant record in storage216 and identify the participant delay offset in the participant record(stored at step 1106). In some examples, the IP attributes may beidentified from the message header. Thus, the IP attributes may beidentified without deep packet inspection. Accordingly, message delayline system may identify some contents of the message (e.g., port and IPaddress, port alone, other message contents) to identify the source ofthe message and extract the participant delay offset from theparticipant record.

At optional step 1114, the message payload may be inspected to identifyone or more TCC-related attributes, for example, by message departuretime module 212. Example TCC-related attributes may include, withoutbeing limited to, a participant identifier (ID), data type (e.g., ordertype), etc. The TCC-related attributes may also be applied to theparticipant delay offset (determined in step 1112), to modify theparticipant delay offset, under some conditions or may selectively applythe participant delay offset. For example, the participant delay offsetmay include more than one value, with each value associated with adifferent TCC-related attribute. In another example, the participantdelay offset may be applied a predetermined weighting based theidentified TCC-related attribute.

At step 1116, message delay line system 104 may apply the participantdelay offset to the received message, as described above with respect toFIGS. 3 and 4.

At optional step 1118, steps 1104-1106 may be repeated periodically, forexample, when active CIP configuration 800 is used with out-of-band TCCmeasurement techniques. Steps 1108-1116 (and optional step 1118) mayrepresent message delay application steps for each participant device102 during participation in message exchange in environment 100.

Next, an example non-limiting scenario for provisioning and delayapplication during message exchange for passive CIP configuration 700 isprovided. A participant identified as “baml” may include equipmentcollocated with downstream system 106 in Mahwah, N.J., for example, inwiring cabinet 5. The participant may request a cross-connect todownstream system 106 (e.g., an exchange system). Downstream system 106may create a LOA with a client cabinet patch panel port (CIP) and adownstream system side patch panel port (DIP) marked in the LOA. On theinstallation of the participant equipment, the TCC propagation delay maybe measured (e.g., by T-BERD), and recorded in a participant record instorage 216 such as: TCC_record:baml_mahwah_cab01_port5, delay 10 usec.Next, the participant record may be expanded with additional IP and portinformation, such as 10.10.10.0/24 src subnet, 172.1.1.1:60010 IP:PORT.

At the start of message exchange (e.g., a start of a trading day),message delay line system 104 may activate the participant record(received from storage 216), and may set up a filter to match theattributes. At a message arrival, message delay line system 104 mayinspect IP:PORT and src_ip information and may identify the participantdelay offset to be applied. Message delay line system 104 may alsoperform processes, such as inspecting the actual message payload, andextracting different attributes in the message payload (e.g., FIX 4.2fields) to determine the participant delay offset that is applied to theparticipant's message.

Referring next to FIG. 12, a flowchart diagram is shown of an examplemethod of determining and applying a participant delay offset tomessages associated with message delay line system 104, according toanother embodiment of the present disclosure. FIG. 12 is described withreference to FIGS. 1, 2 and 8. FIG. 12 illustrates determination of theparticipant delay offset for an active CIP configuration 800 usingin-band TCC delay measurement techniques.

At step 1200, a participant record for a particular participant device102 (e.g., participant device 102-1) may be created, for example, at atime of provisioning the particular participant device 102, similar tostep 1100 (FIG. 11). At step 1202, internet protocol (IP) attributesassociated with the participant device 102 (e.g., participant device102-1) may be determined, at the provisioning time, and stored in theparticipant record, similar to step 1102 (FIG. 11). At step 1202, anapplication platform (e.g., at downstream system 106 or message delayline system 104) may also allocate a unique port (e.g., IP:PORT) perparticipant associated with the application platform.

At step 1204, message delay line system 104 (e.g., via controller 214)may create a filter to match IP attributes to a particular participantand participant record. At step 1206, message delay line system 104, viainput interface 202, may receive an electronic message from amongparticipant devices 102. The received message may include a remotetimestamp applied to the message by active CIP component 808 atparticipant space 802.

At step 1208, message delay line system (e.g., message departure timemodule 212) may determine the TCC propagation delay by comparing thelocal timestamp (applied to the received message by timestamp module 210via wall clock 204) to the remote timestamp included in the receivedmessage (applied by active CIP component 808). In other words, adifference between the local timestamp and the remote timestamp may beused to measure the TCC propagation delay for the message associatedwith a participant. Message departure time module 212 may store the TCCpropagation delay in the participant record (in storage 218) as theparticipant delay offset.

At optional step 1210, the message payload may be inspected to identifyone or more TCC-related attributes, for example, by message departuretime module 212, to modify and/or select the participant delay offset,similar to step 1114 (FIG. 11). At step 1212, message departure timemodule 212 may apply the participant delay offset to the receivedmessage. Thus, with in-band techniques, message delay line system 104may measure the TCC propagation delay “on-the-fly,” based on the localand remote timestamps with each received message.

Next, an example non-limiting scenario for provisioning and delayapplication during message exchange for active CIP configuration 800with in-band delay measurement techniques is provided. A participantidentified as “baml” may include equipment collocated with downstreamsystem 106 in Mahwah, for example, in wiring cabinet 5. The participantmay request a cross-connect to downstream system 106 (e.g., an exchangesystem). Downstream system 106 may provide participant baml with an NICcard (e.g., an active CIP component 808) 808) for communications withthe downstream system 106. Every packet (message) passing through theNIC card may be time-stamped, such that at the reception of the packetat downstream system 106, message delay line system 104 may use thelocal time source (e.g., wall clock 204) to calculate the TCCpropagation delay, based on the local and remote timestamps, and mayidentify the participant delay offset that is applied to the message ofthe participant. Message delay line system 104 may also performprocesses, such as inspecting the actual message payload, and extractingdifferent attributes in the message payload (e.g., FIX 4.2 fields) todetermine the participant delay offset that is applied to theparticipant's message.

Systems and methods of the present disclosure may include and/or may beimplemented by one or more specialized computers or other suitablecomponents including specialized hardware and/or software components.For purposes of this disclosure, a specialized computer may be aprogrammable machine capable of performing arithmetic and/or logicaloperations and specially programmed to perform the functions describedherein. In some embodiments, computers may comprise processors,memories, data storage devices, and/or other commonly known or novelcomponents. These components may be connected physically or throughnetwork or wireless links. Computers may also comprise software whichmay direct the operations of the aforementioned components. Computersmay be referred to with terms that are commonly used by those ofordinary skill in the relevant arts, such as servers, personal computers(PCs), mobile devices, and other terms. It will be understood by thoseof ordinary skill that those terms used herein are interchangeable, andany special purpose computer capable of performing the describedfunctions may be used.

Computers may be linked to one another via one or more networks. Anetwork may be any plurality of completely or partially interconnectedcomputers wherein some or all of the computers are able to communicatewith one another. It will be understood by those of ordinary skill thatconnections between computers may be wired in some cases (e.g., viawired TCP connection or other wired connection) and/or may be wireless(e.g., via a WiFi network connection). Any connection through which atleast two computers may exchange data can be the basis of a network.Furthermore, separate networks may be able to be interconnected suchthat one or more computers within one network may communicate with oneor more computers in another network. In such a case, the plurality ofseparate networks may optionally be considered to be a single network.

The term “computer” shall refer to any electronic device or devices,including those having capabilities to be utilized in connection with anelectronic exchange system, such as any device capable of receiving,transmitting, processing and/or using data and information. The computermay comprise a server, a processor, a microprocessor, a personalcomputer, such as a laptop, palm PC, desktop or workstation, a networkserver, a mainframe, an electronic wired or wireless device, such as forexample, a telephone, a cellular telephone, a personal digitalassistant, a smartphone, an interactive television, such as for example,a television adapted to be connected to the Internet or an electronicdevice adapted for use with a television, an electronic pager or anyother computing and/or communication device.

The term “network” shall refer to any type of network or networks,including those capable of being utilized in connection with the messagedelay line system and downstream system described herein, such as, forexample, any public and/or private networks, including, for instance,the Internet, an intranet, or an extranet, any wired or wirelessnetworks or combinations thereof.

The term “computer-readable storage medium” should be taken to include asingle medium or multiple media that store one or more sets ofinstructions. The term “computer-readable storage medium” shall also betaken to include any medium that is capable of storing or encoding a setof instructions for execution by the machine and that causes the machineto perform any one or more of the methodologies of the presentdisclosure.

FIG. 13 illustrates a functional block diagram of a machine in theexample form of computer system 1300 within which a set of instructionsfor causing the machine to perform any one or more of the methodologies,processes or functions discussed herein may be executed. In someexamples, the machine may be connected (e.g., networked) to othermachines as described above. The machine may operate in the capacity ofa server or a client machine in a client-server network environment, oras a peer machine in a peer-to-peer (or distributed) networkenvironment. The machine may be any special-purpose machine capable ofexecuting a set of instructions (sequential or otherwise) that specifyactions to be taken by that machine for performing the functionsdescribe herein. Further, while only a single machine is illustrated,the term “machine” shall also be taken to include any collection ofmachines that individually or jointly execute a set (or multiple sets)of instructions to perform any one or more of the methodologiesdiscussed herein. In some examples, participant devices 102, messagedelay line system 104 and/or downstream system 106 (FIG. 1) may beimplemented by the example machine shown in FIG. 13 (or a combination oftwo or more of such machines).

Example computer system 1300 may include processing device 1302, memory1306, data storage device 1310 and communication interface 1312, whichmay communicate with each other via data and control bus 1318. In someexamples, computer system 1300 may also include display device 1314and/or user interface 1316.

Processing device 1302 may include, without being limited to, amicroprocessor, a central processing unit, an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA), adigital signal processor (DSP) and/or a network processor. Processingdevice 1302 may be configured to execute processing logic 1304 forperforming the operations described herein. In general, processingdevice 1302 may include any suitable special-purpose processing devicespecially programmed with processing logic 1304 to perform theoperations described herein.

Memory 1306 may include, for example, without being limited to, at leastone of a read-only memory (ROM), a random access memory (RAM), a flashmemory, a dynamic RAM (DRAM) and a static RAM (SRAM), storingcomputer-readable instructions 1308 executable by processing device1302. In general, memory 1306 may include any suitable non-transitorycomputer readable storage medium storing computer-readable instructions1308 executable by processing device 1302 for performing the operationsdescribed herein. Although one memory device 1308 is illustrated in FIG.13, in some examples, computer system 1300 may include two or morememory devices (e.g., dynamic memory and static memory).

Computer system 1300 may include communication interface device 1312,for direct communication with other computers (including wired and/orwireless communication) and/or for communication with a network. In someexamples, computer system 1300 may include display device 1314 (e.g., aliquid crystal display (LCD), a touch sensitive display, etc.). In someexamples, computer system 1300 may include user interface 1316 (e.g., analphanumeric input device, a cursor control device, etc.).

In some examples, computer system 1300 may include data storage device1310 storing instructions (e.g., software) for performing any one ormore of the functions described herein. Data storage device 1310 mayinclude any suitable non-transitory computer-readable storage medium,including, without being limited to, solid-state memories, optical mediaand magnetic media.

While the present disclosure has been discussed in terms of certainembodiments, it should be appreciated that the present disclosure is notso limited. The embodiments are explained herein by way of example, andthere are numerous modifications, variations and other embodiments thatmay be employed that would still be within the scope of the presentdisclosure.

The invention claimed is:
 1. An electronic message transmission delaysystem comprising: one or more computing devices including anon-transitory memory and at least one processor executingcomputer-readable instructions stored in said non-transitory memory; aninput interface receiving a plurality of electronic messages from amongthe one or more sender entities via at least one network; a messagedelay component comprising a delay line element applying one or moredelay offsets to at least one of the electronic messages; and an outputinterface transmitting the plurality of electronic messages to theirrespective destinations via the at least one network, wherein applyingthe one or more delay offsets to the at least one electronic messagecauses said at least one electronic message and at least one otherelectronic message from among the plurality of electronic messages toarrive at their respective destinations at a same or similar time. 2.The system of claim 1, wherein a corresponding delay offset is measured,for a first entity among the one or more sender entities, based on atotal cross-connect (TCC) propagation delay between a client ingresspoint (CIP) component associated with said first entity and a downstreamingress point (DIP) associated with a second entity associated with adestination.
 3. The system of claim 2, wherein the corresponding delayoffset is measured, for each of the plurality of electronic messages,between the CIP component and the DIP based on an in-band measurementtechnique.
 4. The system of claim 3, wherein the in-band measurementtechnique comprises determining the corresponding delay offset based ona difference between a remote timestamp applied to each of saidplurality of electronic messages by the CIP component and a localtimestamp applied by the message delay component.
 5. The system of claim2, wherein the corresponding delay offset is periodically measuredbetween the CIP component and the DIP based on an out-of-bandmeasurement technique.
 6. The system of claim 1, wherein the delay lineelement comprises at least one of a software delay line element (SDLE)and a hardware delay line element (HDLE).
 7. The system of claim 6,wherein the delay line element comprises a SDLE, said SDLE configured torearrange said plurality of electronic messages in a message queueaccording to a respective departure time of each of said plurality ofelectronic messages.
 8. The system of claim 6, wherein the delay lineelement comprises an HDLE, said HDLE configured to apply ahardware-based delay corresponding to a respective departure time toeach of said plurality of electronic messages according to theirrespective sender entities.
 9. The system of claim 1, wherein the one ormore delay offsets comprise: a common predefined delay time, generatedby the system, applied to all of said plurality of electronic messages;a first delay offset, generated by the system, applied to the at leastone electronic message; and a second delay offset that is different fromthe first delay offset, generated by the system, applied to the at leastone other electronic message.
 10. The system of claim 9, wherein atleast one of the common predefined delay, the first delay offset and thesecond delay offset are generated based on at least one of sender entitygeographical location, transmission route, communication medium, networkpropagation characteristics and messaging attributes.
 11. The system ofclaim 9, wherein the message delay component comprises: a messagedeparture time module determining a departure time, from said system,for each of the plurality electronic messages based on the commonpredefined delay time common to all of said plurality of electronicmessages; a message queue storing the plurality of electronic messagespositioned in the message queue according to their respective departuretimes; and a message queue controller transmitting each of the pluralityof electronic messages to their respective destinations according totheir respective departure times via the message queue.
 12. The systemof claim 1, wherein the at least one electronic message and the at leastone other are received by the input interface at different arrivaltimes.
 13. The system of claim 1, wherein the respective destinations ofthe at least one message and the at least one other electronic messagecomprise different geographic locations.
 14. The system of claim 1,wherein the message delay component further comprises: a timestampmodule applying a timestamp to each of the plurality of electronicmessages at the input interface, wherein the message departure timemodule determines, for each of the plurality of electronic messages,departure times from said system based at least in part on therespective applied timestamps.
 15. A computer-implemented method ofdelaying electronic message transmissions comprising: providing a systemcomprising one or more computing devices including a non-transitorymemory, at least one processor executing computer-readable instructionsstored in said non-transitory memory, an input interface, an outputinterface and a message delay component; receiving, by the inputinterface, a plurality of electronic messages from among the one or moresender entities via at least one network; applying, by the message delaycomponent, one or more delay offsets to at least one of the electronicmessages; and transmitting, by the output interface, the plurality ofelectronic messages to their respective destinations via the at leastone network, wherein applying the one or more delay offsets to the atleast one electronic message causes said at least one electronic messageand at least one other electronic message from among the plurality ofelectronic messages to arrive at their respective destinations at a sameor similar time.
 16. The method of claim 15, further comprisingmeasuring a corresponding delay offset, for a first entity among the oneor more sender entities, based on a total cross-connect (TCC)propagation delay between a client ingress point (CIP) componentassociated with said first entity and a downstream ingress point (DIP)associated with a second entity associated with a destination.
 17. Themethod of claim 16, further comprising measuring the corresponding delayoffset, for each of the plurality of electronic messages, between theCIP component and the DIP based on an in-band measurement technique. 18.The method of claim 17, wherein the in-band measurement techniquecomprises determining the corresponding delay offset based on adifference between a remote timestamp applied to each of said pluralityof electronic messages by the CIP component and a local timestampapplied by the message delay component.
 19. The method of claim 16,further comprising measuring the corresponding delay offset,periodically, between the CIP component and the DIP based on anout-of-band measurement technique.
 20. The method of claim 15, whereinthe message delay component comprises a delay line element, said delayline element comprising at least one of a software delay line element(SDLE) and a hardware delay line element (HDLE).
 21. The method of claim20, wherein the delay line element comprises a SDLE, the method furthercomprising: rearranging, by the SDLE, said plurality of electronicmessages in a message queue according to a respective departure time ofeach of said plurality of electronic messages.
 22. The method of claim20, wherein the delay line element comprises an HDLE, the method furthercomprising: applying, by the HDLE, a hardware-based delay correspondingto a respective departure time to each of said plurality of electronicmessages according to their respective sender entities.
 23. The methodof claim 15, wherein applying the one or more delay offsets comprises:generating, by the system, a common predefined delay time and applying,by the message delay component, the common predefined delay time to allof said plurality of electronic messages; generating, by the system, afirst delay offset and applying, by the message delay component, thefirst delay offset to the at least one electronic message; andgenerating, by the system, a second delay offset that is different fromthe first delay offset and applying, by the message delay component, thesecond delay offset to the at least one other electronic message. 24.The method of claim 23, wherein at least one of the common predefineddelay, the first delay offset and the second delay offset are generatedbased on at least one of sender entity geographical location,transmission route, communication medium, network propagationcharacteristics and messaging attributes.
 25. The method of claim 23,wherein the message delay component comprises a message departure timemodule, a message queue and a message queue controller, the methodfurther comprising: determining, by the message departure time module, adeparture time from said system, for each of the plurality electronicmessages based on the common predefined delay time common to all of saidplurality of electronic messages; storing, by the message queue, theplurality of electronic messages, each positioned in the message queueaccording to their respective departure times; and transmitting, by themessage queue controller, each of the plurality of electronic messagesto their respective destinations according to their respective departuretimes and positions in the message queue.
 26. The method of claim 15,wherein the at least one electronic message and the at least one otherare received by the input interface at different arrival times.
 27. Themethod of claim 15, wherein the respective destinations of the at leastone message and the at least one other electronic message comprisedifferent geographic locations.
 28. The method of claim 15, wherein themessage delay component further comprises a timestamp module, the methodfurther comprising: applying, by the timestamp module, a timestamp toeach of the plurality of electronic messages at the input interface,wherein the message departure time module determines, for each of theplurality of electronic messages, departure times from said system basedat least in part on the respective applied timestamps.